Development of actuated Tangible User Interfaces: new interaction concepts and evaluation methods

Riedenklau E. Development of actuated Tangible User Interfaces: new interaction concepts and evaluation methods. Bielefeld: Universität Bielefeld; 2016.Making information understandable and literally graspable is the main goal of tangible interaction research. By giving digital data physical representations (Tangible User Interface Objects, or TUIOs), they can be used and manipulated like everyday objects with the users’ natural manipulation skills. Such physical interaction is basically of uni-directional kind, directed from the user to the system, limiting the possible interaction patterns. In other words, the system has no means to actively support the physical interaction. Within the frame of tabletop tangible user interfaces, this problem was addressed by the introduction of actuated TUIOs, that are controllable by the system. Within the frame of this thesis, we present the development of our own actuated TUIOs and address multiple interaction concepts we identified as research gaps in literature on actuated Tangible User Interfaces (TUIs). Gestural interaction is a natural means for humans to non-verbally communicate using their hands. TUIs should be able to support gestural interaction, since our hands are already heavily involved in the interaction. This has rarely been investigated in literature. For a tangible social network client application, we investigate two methods for collecting user-defined gestures that our system should be able to interpret for triggering actions. Versatile systems often understand a wide palette of commands. Another approach for triggering actions is the use of menus. We explore the design space of menu metaphors used in TUIs and present our own actuated dial-based approach. Rich interaction modalities may support the understandability of the represented data and make the interaction with them more appealing, but also mean high demands on real-time precessing. We highlight new research directions for integrated feature rich and multi-modal interaction, such as graphical display, sound output, tactile feedback, our actuated menu and automatically maintained relations between actuated TUIOs within a remote collaboration application. We also tackle the introduction of further sophisticated measures for the evaluation of TUIs to provide further evidence to the theories on tangible interaction. We tested our enhanced measures within a comparative study. Since one of the key factors in effective manual interaction is speed, we benchmarked both the human hand’s manipulation speed and compare it with the capabilities of our own implementation of actuated TUIOs and the systems described in literature. After briefly discussing applications that lie beyond the scope of this thesis, we conclude with a collection of design guidelines gathered in the course of this work and integrate them together with our findings into a larger frame

TRecS: Ein tangibles, rekonfigurierbares System zur explorativen Datenanalyse

Dawin JR, Hartwig D, Kudenko A, Riedenklau E. TRecS: Ein tangibles, rekonfigurierbares System zur explorativen Datenanalyse. Bielefeld, Germany: Faculty of Technology, Bielefeld University; 2008.Durch die Virtualisierung von Information verlieren Menschen das Gefühl für die Beschaffenheit dieser digitalen Information. Z.B. wird der Festplattenspeicher lediglich in einer Zahl ausgedrückt. Da sich die Maße der Festplatte nicht ändern, wenn Daten auf ihr gespeichert werden, kann man nur anhand dieser virtuellen Zahl den freien Speicherplatz ermitteln. Digitale Informationen sind folglich nicht anfassbar und werden durch das Verlieren physischer Attribute durch Digitalisierung immer abstrakter und unverständlicher. Ein weiteres Beispiel sei Geld. Viele Menschen zahlen bevorzugt mit Kreditkarten, dem sog. "Plastikgeld". Sie kaufen damit ein und merken u.U. gar nicht, dass sie immer mehr Schulden anhäufen. Durch die Nutzung von realem Papier- und Münzgeld hat man ein besseres Verständnis für den eigentlichen Wert des Geldes - auch und gerade durch das Anfassen. Dinge anzufassen, um sie zu begreifen, d.h. zu verstehen, ist sehr wichtig für ein tiefes Verständnis der Dinge. Durch das 'Begreifbar'-machen von Information (oder zumindest das Erfahren und Erkunden) wird eine reale Verbindung zwischen Mensch und Information hergestellt. So ist es durch Anfassen und Manipulieren möglich, etwas über die Beschaffenheit der Dinge zu erfahren. Dies ist die Quintessenz von Tangible Computing. Tangible Computing versucht zwischen der realen, physischen und der virtuellen, digitalen Welt zu vermitteln und die Vorteile beider zu nutzen, um Informationen besser verständlich zu machen. Wegen der intuitiven Benutzbarkeit, die bei der Nutzung von Tangible Computing entsteht, wurde es bereits in vielen Anwendungsgebieten und Domänen genutzt. Beispiele sind klangverarbeitende Systeme (ein sehr prominentes Gebiet für Tangible Computing; REACtable und AudioPad sind nur wenige Beispiele), Datenaustausch, Arbeitsplatzerweiterungen oder Alltagsinterfaces, wie z.B. die sog. "Marble Answering Machine" (Murmel-Anrufbeantworter). In allen uns bekannten Implementationen wird Tangible Computing für eine bestimmte Aufgabe und zumindest ein bestimmtes Anwendungsgebiet genutzt. In dieser Arbeit entwickelten wir eine neuartige Strategie, um Tangible Computing für ein möglichst weites Feld von Anwendungsgebieten nutzbar zu machen. Als Domäne für unser System haben wir tabellarische Datensätze gewählt. Normalerweise werden Daten in einer Datenbank oder das Resultat einer Anfrage an eine relationale Datenbank in tabellarischer Form organisiert. Sogar Zeitserien werden tabellarisch organisiert, indem man pro Zeile einen diskreten Zeitpunkt abbildet. Ganz allgemein können auf diese Weise sehr hochdimensionale, multivariate Daten organisiert und verarbeitet werden. Also kann unser System mit (fast) allen Arten von Daten umgehen. Natürlich gibt es auch Domänen, die in diese Repräsentation nicht eingepasst werden können. Die Domäne Text ist ein Beispiel hierfür. Schriftstücke lassen sich nicht in tabellarische Form pressen, ohne Informationen zu verlieren oder zu verändern

TAOs - Tangible Active Objects for table-top interaction

Riedenklau E. TAOs - Tangible Active Objects for table-top interaction. Bielefeld (Germany): Bielefeld University; 2009.Personal Computers have arrived in almost every part of our life to do work faster and better. They are used for writing texts, creating music or drawings, or simply organizing and guiding everyday tasks. Nearly all these tasks are done with computers which are operated using screen, keyboard and mouse even if their use may be sometimes cumbersome or even unsuitable for some tasks. Human Computer Interaction (HCI) aims to analyze the way people use computers and suggest new methods for interaction. One area of this research field is called "Tangible Interaction". Tangible Interaction tries to use everyday objects as tangible representations for digital data. It is hoped that by pulling the data into the tangible real world (in contrast to the virtual world) they can be made more vivid and graspable and thereby better understandable. These real-world representations are called Tangible User Interface Objects (TUIOs) and the systems they are used in Tangible User Interfaces (TUIs). The main goal of this work is to create active objects as a new kind of TUIO. These active objects extend the concept of TUIOs by the possibility to be not only manipulated by the user but also by the computer. Many different ways of manipulation are possible, e.g. adding LEDs or liquid crystal displays, sound output or tactile and haptic feedback with vibration, etc. One of the most challenging manipulation possibilities is computer controlled planar movement for instance on a desk surface, which will be developed in this work. The developed objects are constructed as modular as possible to be open for future extensions and modifications. A software structure for the coordination of the objects is implemented. Furthermore some applications shown to give examples for the potential of this novel technique

Diplomarbeit: TAOs - Tangible Active Objects for Table-top Interaction

Riedenklau E. Diplomarbeit: TAOs - Tangible Active Objects for Table-top Interaction. Bielefeld: Faculty of Technology, Bielefeld University; 2009.Personal Computers have arrived in almost every part of our live to do work faster and better. They are used for writing texts, creating music or drawings, or simply organizing and guiding everyday tasks. Nearly all these tasks are done with computers which are operated using screen, keyboard and mouse even if their use may be sometimes cumbersome or even unsuitable for some tasks. Human Computer Interaction (HCI) aims to analyze the way people use computers and suggest new methods for interaction. One area of this research field is called 'Tangible Interaction'. Tangible Interaction tries to use everyday objects as tangible representations for digital data. It is hoped that by pulling the data into the tangible real world (in contrast to the virtual world) they can be made more vivid and graspable and thereby better understandable. These real-world representations are called Tangible User Interface Objects (TUIOs) and the systems they are used in Tangible User Interfaces (TUIs). The main goal of this work is to create active objects as a new kind of TUIO. These active objects extend the concept of TUIOs by the possibility to be not only manipulated by the user but also by the computer. Many different ways of manipulation are possible, e.g. adding LEDs or liquid crystal displays, sound output or tactile and haptic feedback with vibration, etc. One of the most challenging manipulation possibilities is computer controlled planar movement for instance on a desk surface, which will be developed in this work. The developed objects are constructed as modular as possible to be open for future extensions and modifications. A software structure for the coordination of the objects is implemented. Furthermore some applications shown to give examples for the potential of this novel technique

Tangible Active Objects and Interactive Sonification as a Scatter Plot Alternative for the Visually Impaired

Riedenklau E, Hermann T, Ritter H. Tangible Active Objects and Interactive Sonification as a Scatter Plot Alternative for the Visually Impaired. In: Brazil E, ed. Proceedings of the 16th International Conference on Auditory Display. International Community for Auditory Display; 2010: 1-7.In this paper we present an approach that enables visually impaired people to explore multivariate data through scatter plots. Our approach combines Tangible Active Objects (TAOs) and Interactive Sonification into a non-visual multi-modal data exploration interface and thereby translates the visual experience of scatter plots into the audio-haptic domain. Our system and the developed sonification techniques are explained in this paper and a first user study is presented

Actuated tangible objects in tabletop interactions

Riedenklau E, Hermann T, Ritter H. Begreifbare sich bewegende Objekte in Tisch-basierten Interaktionsszenarien. i-com. 2012;11(2):20-25.Tangible Interaction is an established subfield of Human-Computer Interaction research. Actuated tangible user interfaces are still an unsufficiently investigated subject, which offers completely new possibilities for interaction design. We present mechanisms for saving and automatically restoring arrangements of tangible objects, two different communication concepts, an exploratory data analysis system for blind users, as well as multi-modal interaction approaches.Begreifbare Interaktion ist ein sich etablierendes Forschungsfeld der Mensch-Maschine-Interaktion. Ein längst noch nicht vollständig untersuchter Zweig dieses Feldes stellen sich selbst bewegende Objekte dar, die völlig neue, bisher nicht erschöpfend untersuchte Interaktionsmöglichkeiten bieten. Vorgestellt werden unter anderem Mechanismen zum Speichern und automatischen Laden von Anordnungen der begreifbaren Objekte, zwei verschiedene Konzepte zur Kommunikation, sowie ein Datenanalysesystem für Blinde und weitere Ansätze für den Einsatz multi-modaler Interaktionstechniken

An integrated multi-modal actuated tangible user interface for distributed collaborative planning

Riedenklau E, Hermann T, Ritter H. An integrated multi-modal actuated tangible user interface for distributed collaborative planning. In: Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction. New York: ACM; 2012: 169-174.<img src="https://pub.uni-bielefeld.de/download/2695603/2702625" width="200" style="float:right;"> In this paper we showcase an integrative approach for our actuated Tangible Active Objects (TAOs), that demonstrates distributed collaboration support to become a versatile and comprehensive dynamic user interface with multi-modal feedback. We incorporated physical actuation, visual projection in 2D and 3D, and vibro-tactile feedback. We demonstrate this approach in a furniture placing scenario where the users can interactively change the furniture model represented by each TAO using a dial-based tangible actuated menu. We demonstrate virtual constraints between our TAOs to automatically maintain spatial relations

Supplementary Material for "An integrated multi-modal actuated tangible user interface for distributed collaborative planning"

Riedenklau E, Hermann T, Ritter H. Supplementary Material for "An integrated multi-modal actuated tangible user interface for distributed collaborative planning". Bielefeld University; 2012.<img src="https://pub.uni-bielefeld.de/download/2695603/2702625" width="200" style="float:right;"> In this paper we showcase an integrative approach for our actuated Tangible Active Objects (TAOs), that demonstrates distributed collaboration support to become a versatile and comprehensive dynamic user interface with multi-modal feedback. We incorporated physical actuation, visual projection in 2D and 3D, and vibro-tactile feedback. We demonstrate this approach in a furniture placing scenario where the users can interactively change the furniture model represented by each TAO using a dial-based tangible actuated menu. We demonstrate virtual constraints between our TAOs to automatically maintain spatial relations. This publication is part of the Tangible Active Objects (TAOs) research project. <video controls="controls" width="100%" height="100%"> <source src="https://pub.uni-bielefeld.de/download/2695603/2695604" type="video/mp4" /> </center

Supplementary Material for "Saving and Restoring Mechanisms for Tangible User Interfaces through Tangible Active Objects"

Riedenklau E, Hermann T, Ritter H. Supplementary Material for "Saving and Restoring Mechanisms for Tangible User Interfaces through Tangible Active Objects". Bielefeld University; 2011.<img src="https://pub.uni-bielefeld.de/download/2696637/2702775" width="200" style="float:right;"> In this paper we present a proof of concept for saving and restoring mechanisms for Tangible User Interfaces (TUIs). We describe our actuated Tangible Active Objects (TAOs) and explain the design which allows equal user access to a dial-based fully tangible actuated menu metaphor. We present a new application extending an existing TUI for interactive sonification of process data with saving and restoring mechanisms and we outline another application proposal for family therapists. <video controls="controls" width="100%" height="100%"> <source src="https://pub.uni-bielefeld.de/download/2696637/2696639" type="video/mp4" /> </center

Tangible Objects and Interactive Sonification as a Scatter Plot Alternative for the Visually Impaired

Presented at the 16th International Conference on Auditory Display (ICAD2010) on June 9-15, 2010 in Washington, DC.In this paper we present an approach that enables visually impaired people to explore multivariate data through scatter plots. Our approach combines Tangible Active Objects (TAOs) and Interactive Sonification into a non-visual multi-modal data exploration interface and thereby translates the visual experience of scatter plots into the audio-haptic domain. Our system and the developed sonification techniques are explained in this paper and a first user study is presented